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Abstract 



This work presents an application that generates real-time holograms to be dis- 
played on a holographic optical tweezers setup; a technique that allows the ma- 
nipulation of particles in the range from micrometres to nanometres. The software 
is written in Java, and uses random binary masks to generate the holograms. It 
allows customization of several parameters that are dependent on the experimental 
setup, such as the specific characteristics of the device displaying the hologram, 
or the presence of aberrations. We evaluate the software's performance and con- 
clude that real-time interaction is achieved. We give our experimental results from 
manipulating 5 jim microspheres using the program. 
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PROGRAM SUMMARY 
Title of program: HoloTrap 

Computer for which the program is designed and others on which it has been 
tested: General computer 

Operating systems or monitors under which the program has been tested: 
Windows, Linux 

Programming language used: Java 

Memory required to execute with typical data: up to 34 Mb including the 

Java Virtual Machine 

No. of bits in a word:8 bits 

Email address: encarni.pleguezuelos@ub.edu (E. Pleguezuelos,). 
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No. of processors used: 1 

Has the code been vectorized or parallelized?: No 

No. of bytes in distributed program, including test data, etc.: 1118 KB 

Distribution format: jar file 

Nature of physical problem: To calculate and display holograms for 
generating multiple and dynamic optical tweezers to be reconfigured 
interactively 

Method of solution: Fast random binary mask for the simultaneous 
codification of multiple phase functions into a phase modulation device 
Typical running time: Up to 10 frames per second 
Unusual features of the program: None 

References: The method for calculating holograms can be found in [M. 
Montes-Usategui, et al. Opt. Express, 14 (2006) 2101-2107.] 



1 Introduction 

In this paper we describe an application that interactively generates multiple 
dynamic holographic optical tweezers. The program allows the user to compute 
holograms displayed in an optical tweezers setup, resulting in trap patterns 
that are reconfigurable in real time. Experimental setup factors are application 
parameters resulting in a completely customizable program. 

Optical tweezers are generated by strongly focusing a laser beam, thus creating 
an optical gradient that traps dielectric particles from micrometric samples 
down to the nanometric scale [1] , due to the transfer of light momentum to the 
trapped particle. This technique has many applications in the manipulation 
of biological samples [2] because it is harmless and the forces involved in 
molecular and biological processes (which are in the same range as the forces 
applied by optical tweezers -about pN) can be measured. 

Holography allows the synthesis of a light wavefront by spatially modifying 
the amplitude and phase of the beam [3] . Knowing how light propagates in the 
setup allows us to determine the relationship between the field amplitudes in 
any two planes along the optical train. In this way, we can calculate the holo- 
gram that reconstructs a desired light distribution on another plane. The use 
of digital holography in optical tweezers provides a lot of flexibility in the de- 
sign of trap patterns. This technique has resulted in the introduction of large 
arrays of optical traps and three-dimensional control [4], [5]. Moreover, the 
shape and properties of the beam can be modified to generate non-Gaussian 
beams such as vortex beams, which are capable of transferring angular mo- 
mentum to the trapped particle [6], or non-diffracting beams [7]. 

Spatial Light Modulators (SLMs), which are used to display digital holograms, 



2 



allow dynamic, computer-controlled modification of the complex transmit- 
tance/ reflectance of the device. The relationship between the sample plane 
and the hologram plane is an inverse Fourier transform, so, in general, the 
hologram is complex. These devices are constrained to display a set of com- 
plex transmittance values, so we should limit our hologram values to those 
available from the modulator. Algorithms have to be designed to find an op- 
timal solution constraining the hologram to the discrete set of values accessi- 
ble. These algorithms are based on iterative methods [8], [9] or on extensive 
search procedures [10]. Both approaches are time consuming and do not allow 
real-time interaction with the sample, since they cannot be calculated and 
displayed as fast as necessary. We recently proposed a method for calculating 
holograms in order to generate optical tweezers. It is based on the random 
mask encoding method for multiplexing phase-only filters [11], and is, to our 
knowledge, the fastest method with 3D control of the trap. This is because it 
is not iterative and the number of operations involved is lower than in other 
direct methods, such as the gratings and lenses (or prisms and lenses) method 
[12]. Another advantage of the algorithm is that it does not produce the ghost 
traps or replicas reported in other methods [13]. 



Other possibilities -such as the generalized phase contrast approach [14] or 
time sharing of the laser beam- allow real-time interaction with the sample, 
but are limited to two-dimensional trap patterns and do not allow genera- 
tion of non-Gaussian beams. The gratings and lenses method has also been 
used to calculate the desired trap pattern for interactive hologram generation 
applications [15], [16]. It has also been proposed direct programming of the 
graphics card, allowing a faster update rate [17]. Other applications are de- 
signed to be used with their commercial setup and do not allow customization 
[18]. 



We present software developed to interact with trapped particles in real time. 
The application calculates and displays the holograms that generate the trap 
pattern according to the user's commands. In section 2 we outline the exper- 
imental setup, emphasizing the aspects that have to be taken into account in 
the software design. The implemented algorithm is explained in section 3. The 
developed application, written in Java™ 2 Platform Standard Edition 5.0 is 
detailed in section 4. The sample plane is be visualized on another monitor, 
using the camera. The camera image can be integrated in the program. We 
show how to do that in section 4.3. However, the camera driver is proprietary 
and cannot be attached due to licensing restrictions. The performance of the 
software and experimental results are given in section 5. 
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2 Holographic optical tweezers 



In optical trapping, a highly focused laser beam exerts gradient forces on the 
sample. Typically, an inverted microscope is modified to focus the beam, while 
still being able to image the sample. Figure 1 shows our experimental setup. 
The laser is a frequency-doubled Nd:YV04 laser from Viasho Technologies. 
The laser beam is expanded and collimated before being reflected by the Spa- 
tial Light Modulator, a HoloEye LCR-2500. On reflection, the SLM modulates 
the phase of the wavefront. The beam size is reduced using an auxiliary tele- 
scopic system (lenses LI and L2 in figure 1, to adapt it to cover the whole of 
the objective's aperture; which is important for stable trapping [1]. The beam 
is fed into the inverted microscope (a Nikon TE2000) through a rear aperture, 
usually used in fluorescence imaging, and focused in the sample plane by the 
microscope objective (a Plan Fluor 100X Nikon oil-immersion objective with 
numerical aperture 1.3). 



Illumination 




Fig. 1. Holographic optical tweezers setup 

The sample is placed at the focal plane of the objective, so the relationship 
between the device and the sample is an inverse Fourier transform except for 
multiplicative phase terms that do not affect our discussion [19]: 

oo 

H(u, v) = JJ g(x, y)e- i ^ T{xu+yv) dxdy, (1) 



where H(u, v) is the hologram, g(x, y) the trap pattern, A the wavelength of the 
light and /' the focal length of the objective. The wavelength, the telescope, the 
modulator and the objective determine the scale factor between our sample 
plane and the hologram. This is left as a parameter in the application, as 
explained in section 4. 

The introduction of the Spatial Light Modulator allows us to design the shape 
of the beam by spatially modifying the amplitude and phase of the light dis- 
tribution in the plane where the modulator is placed. There are different kinds 
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of SLM, such as liquid crystal displays (LCDs) in which the grey level sent 
to each pixel of the modulator is translated into a change in amplitude and 
phase of the incident beam at that point, thus achieving spatial control of 
the light distribution. The modulation also depends on the polarization of the 
input and output light. It is necessary to know the modulation response for 
each grey level. This can be achieved by characterizing the device modulation 
with the polarization conditions [20] in which it will be used. The most com- 
mon configuration is phase-only modulation, which has the least amplitude 
variation. 

LCDs are unable to modulate the whole complex plane [21]. Figure 2 shows 
the experimental characterization of the LCD we use, a HoloEye LCR-2500. It 
shows the complex transmittance value that corresponds to each grey level. It 
is almost a phase modulation from to 2n, although there is a small amplitude 
modulation. The hologram values have to be built using the available modula- 
tion values. To do this, the minimum Euclidean distance between the phase in 
each pixel and the available phase values is calculated, and the nearest phase 
modulation value is used to display the theoretical hologram value. 




-1,0-1 



Fig. 2. Experimental complex modulation values of the SLM as a function of the 
gray level 



To summarize, our synthesized hologram is a grey-level image that results in 
a local modification of the phase of the incident wavefront, and will generate 
a given trapping pattern in the focal plane of the objective (where the sample 
is located). 
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3 Fast method of hologram calculation 



In order to generate holograms in real time we have to use direct methods of 
calculation as opposed to iterative methods. Direct methods consist of gener- 
ating the hologram from analytic solutions, that is, determining analytically 
the inverse Fourier transform of the trap pattern taking into account that one 
is limited in general to phase functions due to modulation constraints. Holo- 
graphically we have the ability to set the three-dimensional position of each 
trap. A trap centred at (a, b) can be described as g(x, y) = 5(x — a,y — b). The 
hologram that generates this distribution is its Fourier transform, that is: 



2tt 

H D (u, v) = J r ~ 1 (g(x, y)) = exp (i^jiu -a + v-b)). (2) 



If the trap is focused at a depth z from the focal plane, the required function 
is a quadratic phase term such as: 

H z (u,v) =exp(t— (u 2 + v 2 )), (3) 

AZ 

whose Fourier transform is another quadratic phase function. 

To generate a vortex, which can transfer angular momentum to the trapped 
particle [22], the following phase function is needed: 

v 

H v (u,v) = exp (i ■ Ztan~ 1(— )), (4) 

u 

this function modifies the wavefront to convert it to a Laguerre-Gaussian 
mode, which carries angular momentum. The quantity I is called the topo- 
logical charge and is related to the orbital angular momentum of each photon 
by L = lh. 

In these examples the solution is a pure phase function, so codifying it us- 
ing phase-only modulation is straightforward: simply choose the closest phase 
given by the device. However, if N traps are required, the hologram is a sum 
of as many phases as traps displayed, resulting in a complex function that 
cannot be directly displayed with a phase-only modulation: 

N 

H(u, v) = J2 ( H Dk + H Lk + H Vk ) ^ exp (i(f)(u, v)). (5) 

k=0 



The method for the codification of the hologram cannot be time-consuming 
if we require it to be implemented in real time. Our approach, more detailed 
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in [11], defines as many different domains I k of the modulator as traps to be 
displayed. Each domain consists of a set of modulator pixels that shows a phase 
function. In this way, each set is in charge of generating a single trap. The 
hologram (equation 6) consists in the multiplication of the phase functions, 
Hk(u, v), (as in equations 2 and 3) by spatially disjoint binary masks, m k {u, v) 
- the set of pixels of the domains I k . 

N 

H(u,v) = J2 m k(u,v) ■ H k (u,v), (6) 
where 

( 1 if (u, v) G I k 

m k {u,v) = t 

I elsewhere. 

The domains I k do not overlap, and together they cover the whole modulator. 
For example, we can generate the domains by randomly deciding which pixels 
will belong to each trap. This is a good choice since the mask that defines 
every sub-hologram is not a geometric function: the convolution of the shape 
of the trap with the Fourier transform of the mask would result in noise 
distributed through the resulting plane [11]. As can be seen, the solution is 
as fast as generating the I k domains each time a trap is added or deleted, 
and computing the arguments of the complex exponentials H k (u, v) to display 
the hologram. Figure 3 shows an example of a hologram in which half of the 
pixels display a linear phase function and the other half a quadratic phase. 
The resulting light distribution would be two different traps placed off-centre, 
at two different depths. 




Fig. 3. Hologram calculated using the random binary masks method 

This method is -to our knowledge- the fastest way to generate simple trap 
patterns. The most common fast method for the generation of optical tweezers 
(the gratings and lenses method) consists of generating the analytic hologram 
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from equation 5 and discarding the amplitude information. This method is 
slower than our random binary masks method because the calculation time 
increases with the number of traps and requires the computation of trigono- 
metric functions [12]. Due to the discarded amplitude information, the in- 
tensity distribution among the traps may be different from that expected. 
Another advantage of our random binary masks method is that the trap pat- 
tern generated does not present replicas [13]. While other methods tend to 
concentrate the energy not located in the traps in bright spots, resulting in 
false traps, the random binary masks method distributes the remaining energy 
in noise, due to the convolution with each random mask. The downside is that 
the efficiency of the traps is lower than that achieved with other methods. 



4 Description of the program 



The software we present calculates and displays holograms to generate opti- 
cal tweezers allowing real-time interaction with samples. Each change in the 
number or position of the trap requires recalculation of the hologram. The 
program responds quickly enough to provide close to video-rate feedback from 
the sample. 

The reason for using Java is that development costs are low. Moreover Java 
allows easy generation of the Graphical User Interface (GUI) and easy inte- 
gration with C++ generated dlls; the most generalized hardware driver dis- 
tribution method. Another advantage of using Java is simple remote control 
of the experiment over the Internet. If the computer controlling the camera 
acts as a web server, you just have to transform the program generated into 
a servlet and use the Remote Method Invocation (RMI) classes. 

The source code is distributed into three different classes. The first class, 
TRBase, generates the GUI and handles the events related to the input pa- 
rameters. It also initiates the second class, PanelCoord, the panel in which the 
user clicks and drags to generate and move a trap, and so this class monitors 
these mouse events and calls to the third class, FrameHolo. This third class 
calculates and displays the hologram using the mouse coordinates and the 
input parameters. The documentation of the application, in which the differ- 
ent classes and implemented methods are detailed, can be found in the folder 
/html zipped within the application jar file. This documentation can be also 
found in our website [23]. 
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4-1 Graphical User Interface 

A screen capture of the GUI can be found in figure 4. This program allows 
user control of several variables and initial data: 

• The hologram size, in pixels. If the size is set to 1024x768, which is our 
SLM full resolution, the hologram is calculated with half the number of and 
zoomed to fill the modulator, reducing computation time. 

• The scale factor between the Spatial Light Modulator plane and the visual- 
ization plane. The scale can be modified by changing the Row and Column 
factors. This scale factor can be found experimentally, or deduced from the 
different experimental parameters: telescope, SLM, objective, and field of 
view [24]. 

• The Init button asks for a file containing a precalculated map of the phase 
modulation and a phase aberration correction (see section 4.2). In our case, 
the aberration is a distortion of the wavefront due to the curvature of the 
modulator surface. There is an example of a phase-only function map and 
a null aberration correction attached in the .jar file to check the required 
format. To run the application using these two ideal condition files, after 
pressing the Init button, just press OK on the dialog box "Use the default 
aberration and modulation files". Each time a hologram is generated, the 
correction is added and then the nearest grey value is assigned using the 
precalculated map. 

• A selector allows you to choose the manipulated trap if more than one trap 
is generated. The selected trap is indicated by a red circle, whereas the 
unselected traps are in green. 

• A slider allows you to modify the trap depth, from -5/im to 5/im. The 
"Depth Factor" text field allows you to modify the available depth range. 

• By changing the integer in the " lvortex" text field (see figure 4) an optical 
vortex carrying angular momentum is generated by adding a vortex phase 
function (equation 4) with the specified topological charge. 

• The Delete trap button deletes the selected trap. This involves a reconfigu- 
ration of the random binary masks, which have to be recalculated. 

• The hologram is calculated by the method selected in the Radio Button. 
The default calculation method is random binary masks, but gratings and 
lenses is also available. 



4-2 Calculating and displaying a hologram 

This section details the computational process of generating a hologram, see 
figure 5. First, the application is initiated and the user enters the desired pa- 
rameters (such as hologram size or scale factors). The central panel in the ap- 
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! Hologram generation 



Hologram generation 




Fig. 4. Application user interface 

plication controls the event handling of the user interaction. When the mouse 
is clicked on the panel, the mouse coordinates are obtained. The random mask 
is calculated, resulting in the whole modulator, because there is one single trap 
in this first step. With the mouse coordinates, a phase grating corresponding 
to the trap is calculated (equation 2) and the hologram is displayed. Each time 
another trap is added, the random binary masks have to be calculated and 
then each set of pixels show the corresponding phase function. If the mouse 
is dragged or the depth slider is moved, the coordinates of the selected trap 
change. A change in the coordinates of a single trap means that only the pixels 
of the mask corresponding to that trap have to be recalculated. 

To generate traps in real time, the algorithm has to be fast, but there are 
also other considerations that affect the performance of the process. We have 
optimized the process of adaptation of the hologram to the modulation values 
by generating a map of the correspondence between all the possible phase 
values between and 2n and the nearest phase given by a grey level. In 
general this is not a linear relationship. The phase value assigned to each grey 
level is stored in a file that the program reads as an initial parameter. An 
example of an ideal phase assignment can be found in the mapJdeal.txt file 
distributed in the jar. Once the analytical phase value is calculated, the map 
provides the grey level to be displayed. An incorrect assignment can cause 
variations in the reconstructed trap pattern. 

Another experimental issue that affects the calculation time is the possible ex- 
istence of aberrations in the optical system, which can be corrected with the 
SLM when generating the trap pattern. In our case, the modulator reflected 
wavefront is distorted because the device is not flat. We can correct this aber- 
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Fig. 5. Program Flowchart 

ration by adding a fixed phase pattern to each hologram. As an example, the 
file phaberr_1024x768.txt is a null aberration correction attached in the jar 
file, that shows the format of the aberration file for the specified hologram 
size. 



In order to ensure a fast response of the displayed hologram, two main fac- 
tors have to be taken into account. First, the hologram generation has to be 
as fast as possible, including algorithm calculation, addition of the aberra- 
tion correction and adaptation to the modulation. Second, speed of access to 
the graphic hardware has to be ensured. Our approach is to take advantage 
of the V olatilelmage class in the Java SDK. The hologram is stored as a 
hardware-accelerated off-screen image, in such a way that rendering opera- 
tions are accelerated through the graphics card. Thus, hologram is displayed 
without using the CPU. This class parallelizes the display of the hologram 
and the calculation processes, with the CPU performing the calculation. 



4-3 Camera control 



The image of the sample can be displayed on another monitor to control ma- 
nipulation. Our program is enhanced if the camera image is incorporated into 
the interactive interface, although it can be used with the image separated 
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from it. In this section we explain how we integrated our video stream man- 
agement, as a guide for users on how to embed their own. We used a QlCam 
Fast 1934 from Qlmaging Corp. [25] camera. It is not compliant with the 
IIDC Digital Camera Specification (DCAM), which is the standard protocol 
Fire Wire cameras should follow, so the SDK provided by the manufacturer had 
to be used. This is a drawback to distributing the camera-integrated version 
of the program, and so a version without a camera accompanies the paper. If 
a DCAM-compliant camera is used, the Java API for FireWire jlibdcl39A [26] 
can be incorporated into the program instead of the camera SDK, making it 
suitable for all DCAM-compliant cameras. 

The Qimaging libraries have to be used with a C++ compiler, so the Java 
Native Interface (JNI) class [27] is needed to embed the camera library into 
the Java structure. JNI is a common trick for gaining compatibility with native 
methods across a Java virtual machine. We need the following native functions: 

public native int initCameraQ; (Detects the camera) 

public native int initGrabQ; (Initiates the recording) 

public native int doGrab(byte[] pix); (Saves the image into a pixel array) 

public native int StopGrabQ; (Stops recording) 

Each native Java method has its corresponding function in C++. The process 
of calling from a Java program code contained in the proprietary library is 
[27]: 

- Declaration of the native methods in the Java application, in our case the 
methods listed above. 

- The loading of the library containing the native code implementation, by 
calling the function System. load(" JNIQCam.dll"), where 'JNIQCam.dll' is 
our generated library name (even it does not exist yet). The Java application 
has to be compiled at this point without being executed. This library is not 
the proprietary library, but one generated by the user, defining what each 
native method does. 

- Generation of the header (.h) file that contains the interface assigning the 
Java methods to the C native functions. As an example, the functions are 
defined in this header as: 

JNIEXPORT jint JNICALL Java_initCamera(JNIEnv *, jobject); 
JNIEXPORT jint JNICALL Java_initGrab(JNIEnv *, jobject); 
JNIEXPORT jint JNICALL Java_doGrab(JNIEnv *, jobject, jbyteAr- 
ray); 

JNIEXPORT jint JNICALL Java_StopGrab(JNIEnv *, jobject); 
This file is the communication channel between the two languages. 

- Creation of the C++ functions. The library (JNIQCam.dll) has to contain 
the C++ source calling to the camera library. As an example, our C++ 
method that disconnects the camera is: 
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JNIEXPORT jint JNICALL Java_tr_StopGrab(JNIEnv *, jobject){ 
delete [] pixels; 
if(hCamera != NULL) 

{ 

QCam_CloseCamera( hCamera ); 
} 

QCam_ReleaseDriver () ; 
return 0; 

} 

In this example we free the image memory through the delete order. The 
calling to CloseCamera(hCamera) frees the hCamera object, hCamera is 
the object initiated in the method initCamera, which contains the camera 
properties and prevents other applications accessing the camera. Next, the 
camera driver is released with the command ReleaseDriver(). 
- Compilation and execution of the code. 

If the user had the same camera, a .dll file should be generated and the com- 
mented lines in the Java source, marked as "/ /Comment if there is no QlCam 
available" , should be uncommented. 



5 Performance of the software 

Figure 4 shows a screen capture of the program. The tests were carried out 
on a Pentium IV HT, 3.2 Ghz, with lite versions of the program, where not 
all the options were available. These lite versions can be obtained from our 
website [23]. 

The speed of the software when generating holograms in response to a mouse 
drag is about 10 fps (frames per second). This value measures the number of 
holograms displayed per second on the modulator. The full resolution sized 
holograms are achieved by calculating holograms of 512x384 pixels and resiz- 
ing them into 1024x768 pixels. The adaptation mapping that we have created 
from the experimental phase modulation values does not affect the speed of 
the hologram generation. In contrast, the inclusion of the aberration correc- 
tion affects slightly the performance by decreasing the hologram calculation 
speed. The time response does not increase with the number of traps, because 
the number of pixels the phases have to be computed for (the number of pixels 
defining each mask) decreases as the number of traps increases. 

Figure 6 shows screen shots of experimental manipulation of polystyrene beads 
of 5/im diameter. A first microsphere is captured and dragged close to another, 
which is trapped and moved with a second trap. 
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Fig. 6. Screen shots showing the experimental manipulation of polystyrene beads, 
5/Uin diameter 

6 Concluding remarks 

We have presented an application for calculating and displaying holograms in 
real time to generate multiple reconfigurable optical tweezers. The applica- 
tion allows the user to generate, delete or modify optical traps interactively. 
We used the random binary masks method because of its speed. The soft- 
ware takes into account different parameters given by the experimental setup, 
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and so it is not limited to a single configuration. The different options have 
been detailed, including the adaptation to the modulation and the correction 
of possible aberrations. The scale factor and the hologram size can also be 
modified. Some strategies for accelerating hologram calculation and display 
are explained. A second version of the program takes advantage of the propri- 
etary libraries of the camera used in order to embed the image provided by 
the camera and the program. The viability of the software is comparable to 
that of other applications in the literature. We include an example of optical 
manipulation using this program. In future work we would like to make the 
software compatible with IICAM-compliant FireWire cameras. 
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